System, method and computer program product for creating a summarization from recorded audio of meetings

ABSTRACT

A meeting summarization method, system, and computer program product, include capturing notes of a user including a time stamp from the user associated with a meeting, synchronizing an agenda of the meeting and the notes of the user based on a correlation between a time stamp of a topic on the agenda and a time stamp of the notes of the user, and analyzing the synchronized topic and the notes to determine highlights of the meeting based on a co-occurrence of the time stamp of the notes of the user and the time stamp of the topic on the agenda.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation application of U.S. patentapplication Ser. No. 15/785,520, filed on Oct. 17, 2017 which isContinuation application of U.S. patent application Ser. No. 15/250,780,filed on Aug. 29, 2016, the entire contents of which are herebyincorporated by reference.

BACKGROUND

The present invention relates generally to a meeting summarizationmethod, and more particularly, but not by way of limitation, to asystem, method, and computer program product for extracting orbookmarking the key segments of the audio recorded during meetings tocreate a meeting summarization.

Often, business meetings arc recorded, so that the audio can be playedback at a later time. A meeting attendee or other interested party maywant to find a particular segment of audio from a prior meeting.However, it can be cumbersome to listen to long segments of audio, justto find the useful pieces of information embedded in the playbackstream.

Conventional meeting summarization techniques consider capturing meetingnotes from multiple attendees and related multimedia in a central place.The summarization is done by linking notes entered into a webapplication with video and other media (e.g. video, slides, etc.)captured and stored on server. The meeting attendees enter the meetingminutes directly in a web form.

However, needs in the art include the needs to automatically extract orbookmark the key segments of the audio recorded during meetings, whichthen may be used, for example, to create a meeting summarization.

SUMMARY

In an exemplary embodiment, the present invention can provide acomputer-implemented meeting summarization method, the method includingrecording meeting audio of a meeting, capturing notes including a timestamp from each of a plurality of users associated with the meeting,synchronizing the recorded meeting audio of the meeting and each of thenotes of each of the plurality of users based on a correlation betweenthe time stamp, and analyzing the synchronized meeting audio and notesto determine highlights of the meeting based on a co-occurrence of notesbetween the plurality of users.

One or more other exemplary embodiments include a computer programproduct and a system.

Other details and embodiments of the invention will be described below,so that the present contribution to the art can be better appreciated.Nonetheless, the invention is not limited in its application to suchdetails, phraseology, terminology, illustrations and/or arrangements setforth in the description or shown in the drawings. Rather, the inventionis capable of embodiments in addition to those described and of beingpracticed and carried out in various ways and should not be regarded aslimiting.

As such, those skilled in the art will appreciate that the conceptionupon which this disclosure is based may readily be utilized as a basisfor the designing of other structures, methods and systems for carryingout the several purposes of the present invention. It is important,therefore, that the claims be regarded as including such equivalentconstructions insofar as they do not depart from the spirit and scope ofthe present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the invention will be better understood from the followingdetailed description of the exemplary embodiments of the invention withreference to the drawings, in which:

FIG. 1 exemplarily shows a high-level flow chart for a meetingsummarization method 100;

FIG. 2 depicts a cloud-computing node 10 according to an embodiment ofthe present invention;

FIG. 3 depicts a cloud-computing environment 50 according to anembodiment of the present invention; and

FIG. 4 depicts abstraction model layers according to an embodiment ofthe present invention.

DETAILED DESCRIPTION

The invention will now be described with reference to FIG. 1-4, in whichlike reference numerals refer to like parts throughout. It is emphasizedthat, according to common practice, the various features of the drawingsare not necessarily to scale. On the contrary, the dimensions of thevarious features can be arbitrarily expanded or reduced for clarity.

With reference now to the example depicted in FIG. 1, the meetingsummarization method 100 includes various steps to aggregate the meetinghighlights and create a condensed meeting summarization. As shown in atleast FIG. 2, one or more computers of a computer system 12 according toan embodiment of the present invention can include a memory 28 havinginstructions stored in a storage system to perform the steps of FIG. 1.

Thus, the meeting summarization method 100 according to an embodiment ofthe present invention may act in a more sophisticated, useful andcognitive manner, giving the impression of cognitive mental abilitiesand processes related to knowledge, attention, memory, judgment andevaluation, reasoning, and advanced computation. A system can be said tobe “cognitive” if it possesses macro-scale properties—perception,goal-oriented behavior, learning/memory and action—that characterizesystems (i.e., humans) generally recognized as cognitive.

Although one or more embodiments (see e.g., FIGS. 2-4) may beimplemented in a cloud environment 50 (see e.g., FIG. 3), it isnonetheless understood that the present invention can be implementedoutside of the cloud environment.

In step 101, meeting audio 140 of a meeting is recorded. The recordingcomprises a time stamp associated with each part of the audio.Preferably, the meeting audio 140 is recorded from a multi-personmeeting using one or more microphones.

In step 102, notes including a time stamp of when the notes were takenfrom each user of the meeting are captured. That is, time-stamped notestaken by meeting attendees are captured such that the time stamp of whenthe notes were taken and the time stamp of the recorded meeting audiocan be synchronized (as described later).

In some embodiments, as an input device, Smart Pens can be used tocapture handwritten notes the user takes using the pen with a time stampof when the notes that were taken. Alternatively, a key stroke loggingsoftware can be utilized to capture a time stamp of when a user entersnotes into an electronic device such as a smart phone, laptop, computer,tablet, etc. In some embodiments, a user may take audio notes via anaudio recording device which can log the time stamp of the audio inputby the user (e.g., if the meeting is an online meeting or the user is ata remote location such that the user speaking does not interrupt themeeting). Further, a whiteboard (or the like) may be used with a cameracapturing the text on the whiteboard.

In other words, in step 102, a time stamp associated with notes iscaptured for notes a user takes by as an input device (e.g., a smartpen, an electronic device, audio recorder, etc.).

In step 103, the recorded audio and each of the notes of the users aresynchronized according to the time stamp. That is, the notes from eachof the users is synchronized with the audio file for the same time(e.g., according to the time stamp) in step 103. For example, all of thenotes taken for each of the users at the five-minute mark of the meetingare synchronized together with the audio from the five-minute mark ofthe meeting.

In some embodiments, the notes of users may correspond to audio from aprevious time stamp (e.g., the user takes notes on what was previouslydiscussed). In step 103, the recorded audio is synchronized with theassociated note-taking since the note-taking may have started after theassociated audio one wants to capture has begun. In one exemplaryembodiment, the synchronization may compensate for the delay between theaudio and the notes associated with the audio by synchronizing all thenotes from a same time stamp with each other and treating thiscollective sample as a bag of words. Then, the sample of the bag ofwords from the notes is matched to the bag of words of the transcribedaudio that preceded the note taking, going back a predetermined amountof time in the transcribed audio (e.g., thirty seconds, two minutes,five minutes, etc.). The bag of words can be used to create a map andidentify the correlation between the audio and the collected notes.Additionally, the transcribed text can be annotated with thecorresponding tone (from a tone analyzer) and sentiment analysis can becarried out on the text to measure the overall affect or mood of thesummary. Since the attendees might have different background orinterest, then this map can be used to classify attendees based on theirbackground, understanding of the content, etc. Moreover, the map can beused in a replay tool to identify the highest note taken segments duringthe meeting. That is, a Natural Language Processor (NPL) may be used toassociate the words in the notes with the words of the audio recordingto synchronize the notes with the part of the audio recording that thenotes were taken for.

In step 104, the synchronized audio recording with the notes is analyzedto determine highlights of the meeting based on a co-occurrence of notesfrom the users. Generally, a “co-occurrence” has a temporal (i.e.,substantially same time) meaning or in some cases may have a contentmeaning (i.e., what the user's are writing if the content is analyzedand related). For example, if multiple user's take notes at a particulartime stamp (e.g., a co-occurrence of notes), a highlight is associatedwith the time stamp that the co-occurrence of note taking occurred. Thatis, regardless of what the notes say, a highlight is determined if thereis a co-occurrence of notes at a same time stamp.

Alternatively, in some embodiments, for example, a highlight maycomprise a profits amount and goals for a quarter of sales based on theaudio recording discussing profits and goals and each of the user notesreferring to the profits and goals for the quarter. That is, a highlightof profits and goals is determined based on a co-occurrence of the notesof multiple users discussing profits and goals as well as the audiorecording discussing the same. However, if the audio recording isdiscussing profits and goals and notes of a user recite “pick up breadat store”, a highlight s not created because “pick up bread at thestore” is not in the audio recording and does not co-occur in otherusers notes. In other words, the synchronized audio and notes frommultiple meeting attendees are analyzed in step 104, looking forco-occurring note taking, to determine the meeting highlights. Thus, ameeting highlight ma be when most notes are taken.

In step 106, each of the highlights determined in step 105 is alignedwith a topic of the meeting or with a topic of a meeting agenda. Forexample, if there is no agenda, the topics of the meeting can beextracted from the audio recording using a NPL. The highlights can bealigned with the topics such that a user can later access the highlightsof the meeting based on a topic. For example, if a user is only part ofthe marketing department but not part of the sales, the user may wishonly to review highlights pertaining to the marketing topics and not thesales topics. Alternatively, the highlights may be aligned to each topiclisted in the agenda. Thus, the users can select the highlights based onthe agenda.

In step 107, a concept associated with the highlight can be identifiedbased on the co-occurring notes. For example, the highlight may be abouta topic of sales, but the concept of the topic may refer to how toincrease sales. Using a NFL the concept of “increasing sales” can beidentified as the concept for the highlight and topic aligned with thehighlight. Thus, “concept” may mean what the meeting is about.

In one exemplary embodiment, the users may be using smart pens withBluetooth communication in a meeting room that is equipped with a hubthat includes a microphone and Bluetooth wireless capabilities.Participants at a meeting are equipped with the pens fir note taking,each with a sensor that detects the flow of ink from the tip and aBluetooth wireless capability (e.g., in this embodiment, the ink strokesare not captured, just the fact that stroking is occurring). When themeeting begins, the host activates the microphone on the hub to recordthe audio of the meeting. Participants are encouraged to turn on thepens, which turns on the ability of the pens to detect inking (e.g.,capturing the notes), When the pen detects the ink beginning to flow, itsends out a Bluetooth lower energy advertisement indicating that thatpen is being used. A similar message is sent when ink stops flowing.When the hub detects an advertisement announcing the use or cessation ofuse of a pen, it mates the tin ae of that event in the timeline of theaudio captured. Subsequent to the meeting, the writing activity isanalyzed and regions of the audio track in proximity to writing activityare tagged. That is, the timestamps of the handwritten notes stored onthe server are analyzed, the application programing interface of thesmart pen is used, and the most frequent time periods of user notetaking activity are determined. The corresponding segments of audio forthose time periods are saved along with handwritten notes and tagged as“highlights” in the meeting repository of the server. A summary of themeeting is a collection of the highlights sorted with the most intensenote taking segments listed first. A user experience is provided thatallows one to playback audio with various levels of details. Low levelaudio summarization includes only audio that occurred at the same timeas the most intense inking activity. Medium level summarization includesaudio capture near even isolated inking events.

Alternatively, instead of Bluetooth, ultrasound may be used. Pens emitan ultrasonic signal when stroking. The hub may listen for the presenceand amplitude of the ultrasonic signal.

In some embodiments, a tablet and/or stylus may be used to take notes bythe user. Meeting participants are equipped with tablets paired eachwith a stylus, network-synchronized clock, microphone, and a note takingapplication. As a meeting begins, each participant takes notes with hisstylus as needed. The note taking application on the tablet capturesstrokes and audio and sends a message to a meeting summarization serverevery time ink stroking occurs. One or more tablets also send audio tothe meeting summarization server either continuously or in proximity toinking events. Each inking notification and submitted audio segmentincludes a time stamp. At the end of the meeting, the host asks thesummarization server to generate a summarization of the meeting audio.

In other exemplary embodiments, a meeting room may be equipped with awhiteboard and a camera directed at the whiteboard. The camera alsoincludes a microphone. At the beginning of the meeting, the host turnson the camera and microphone. A meeting summarization server monitorsthe audio and images captured by the camera. When the meetingsummarization server detects a writing on or erasure of the whiteboard,it notes when this occurs in a timeline. At the end of the meeting, thehost asks the meeting summarization server for a meeting summary. Theserver generates a summary by analyzing the video stream for whiteboardinking/erasing activity and including primarily segments that includethat activity.

Thereby, the notes of each meeting attendee can be analyzed to determinethe time stamps of the “meeting Highlights”. For example, if the speakersays something particularly poignant, it is likely that most of theattendees will be taking notes at that moment. Several of these momentscan then be extracted from the audio of a long meeting, to, e.g., createa summary of the meeting highlights. Alternatively, or in addition, themethod 100 may identify the segments of the meeting of special notes,isolate the associated audio streams, and either from the notes or theactual audio transcript, provide a summarization, and for eachsummarization a link to the associated audio segment. Additionally, ifthere is a meeting agenda, the method 100 may automatically try toassociate these especially noteworthy incidents to items on the agenda(e.g. by doing a word similarity assessment between the words in theagenda item and words in the extracted text and words in the associatednotes). Finally, when the meeting notes are hosted on the Cloud, thenotes can be augmented with related concepts aligned to the central setof concepts that are being discussed to assist the attendees inunderstanding concepts that they might otherwise find difficult.

Exemplary Aspects, Using a Cloud-Computing Environment

Although this detailed description includes an exemplary embodiment ofthe present invention in a cloud-computing environment, it is to beunderstood that implementation of the teachings recited herein are notlimited to such a cloud-computing environment. Rather, embodiments ofthe present invention are capable of being implemented in conjunctionwith any other type of computing environment now known or laterdeveloped.

Cloud-computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as Follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at sonic level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as Follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client circuits through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud, infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as Follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud-computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud-computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 2, a schematic of an example of a cloud-computingnode is shown. Cloud-computing node 10 is only one example of a suitablenode and is not intended to suggest any limitation as to the scope ofuse or functionality of embodiments of the invention described herein.Regardless, cloud-computing node 10 is capable of being implementedand/or performing any of the functionality set forth herein.

Although cloud-computing node 10 is depicted as a computer system/server12, it is understood to be operational with numerous other generalpurpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with computersystem/server 12 include, but are not limited to, personal computersystems, computer systems, thin clients, thick clients, hand-held orlaptop circuits, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputer systems, mainframe computer systems, and distributedcloud-computing environments that include any of the above systems orcircuits, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes, Computer system/server 12 may be practiced in distributedcloud-computing environments where tasks are performed by remoteprocessing circuits that are linked through a communications network. Ina distributed cloud-computing environment, program modules may belocated in both local and remote computer system storage media includingmemory storage circuits.

Referring again to FIG. 2, computer system/server 12 is shown in theform of a general-purpose computing circuit. The components of computersystem/server 12 may include, but are not limited to, one or moreprocessors or processing units 16, a system memory 28, and a bus 18 thatcouples various system components including system memory 28 toprocessor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externalcircuits 14 such as a keyboard, a pointing circuit, a display 24, etc.;one or more circuits that. enable a user to interact with computersystem/server 12; and/or any circuits (e.g., network card, modem, etc.)that enable computer system server 12 to communicate with one or moreother computing circuits. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,circuit drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Referring now to FIG. 3, illustrative cloud-computing environment 50 isdepicted. As shown, cloud-computing environment 50 comprises one or morecloud-computing nodes 10 with which local computing circuits used bycloud consumers, such as, for example, personal digital assistant (FDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud-computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingcircuit. It is understood that the types of computing circuits 54A-Nshown in FIG. 3 are intended to be illustrative only and that computingnodes 10 and cloud-computing environment 50 can communicate with anytype of computerized circuit over any type of network and/or networkaddressable connection (e.g., using a web browser).

Referring now to FIG. 4, an exemplary set of functional abstractionlayers provided by cloud-computing environment 50 (FIG. 3) is shown. Itshould be understood in advance that the components, layers, andfunctions shown in FIG. 4 are intended to be illustrative only andembodiments of the invention are not limited thereto. As depicted, thefollowing layers and corresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage circuits 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, manage tilt layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud-computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within thecloud-computing environment, and billing or invoicing for consumption ofthese resources. In one example, these resources may compriseapplication software licenses. Security provides identity verificationfor cloud consumers and tasks, as well as protection for data and otherresources. User portal 83 provides access to the cloud-computingenvironment for consumers and system administrators. Service levelmanagement 84 provides cloud-computing resource allocation andmanagement such that required service levels are met. Service LevelAgreement (SLA) planning and fulfillment 85 provide pre-arrangement for,and procurement of, cloud-computing resources for which a futurerequirement is anticipated in accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud-computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and, more particularly relative to thepresent invention, the meeting summarization method 100.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer-readable storagemedium (or media) having computer-readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer-readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer-readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer-readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer-readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer-readable program instructions described herein can bedownloaded to respective computing/processing devices from acomputer-readable storage medium or to an external computer or externalstorage device via a network, for example, the Internet, a local areanetwork, a wide area network and/or a wireless network. The network maycomprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer-readable programinstructions from the network and forwards the computer-readable programinstructions for storage in a computer-readable storage medium withinthe respective computing/processing device.

Computer-readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer-readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer-readable program instructions by utilizing state information ofthe computer-readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer-readable program instructions.

These computer-readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer-readable program instructionsmay also be stored in a computer-readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that thecomputer-readable storage medium having instructions stored thereincomprises an article of manufacture including instructions whichimplement aspects of the function/act specified in the flowchart and/orblock diagram block or blocks.

The computer-readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

Further, Applicant's intent is to encompass the equivalents of all claimelements, and no amendment to any claim of the present applicationshould be construed as a disclaimer of any interest in or right to anequivalent of any element or feature of the amended claim.

What is claimed is:
 1. A computer-implemented meeting summarizationmethod, the method comprising: capturing notes of a user including atime stamp from the user associated with a meeting; synchronizing anagenda of the meeting and the notes of the user based on a correlationbetween a time stamp of a topic on the agenda and a time stamp of thenotes of the user; and analyzing the synchronized topic and the notes todetermine highlights of the meeting based on a co-occurrence of the timestamp of the notes of the user and the time stamp of the topic on theagenda.
 2. The computer-implemented method of claim 1, wherein theanalyzing further analyzes a concept of the co-occurring notes todetermine the highlights only if the concept of each of theco-occurrence of the time stamp of the agenda and the time stamp of thenotes have an overlap with each other greater than a predeterminedthreshold value.
 3. The computer-implemented method of claim 1, furthercomprising creating a summary of the highlights of the meeting byaggregating the co-occurring notes for each of the plurality of users.4. The computer-implemented method of claim 1, wherein the highlightsare determined according only to a match of the time stamp of theco-occurrence of notes.
 5. The computer-implemented method of claim 1,further comprising identifying a concept associated with each of thehighlights from processing a recorded audio of the meeting to create asummary of the meeting based on concepts.
 6. The computer-implementedmethod of claim 1, wherein the co-occurrence is determined based on atleast two of the plurality of users taking notes at a same time stamp.7. The computer-implemented method of clan 1, embodied in acloud-computing environment.
 8. A computer program product for meetingsummarization, the computer program product comprising acomputer-readable storage medium having program instructions embodiedtherewith, the program instructions executable by a computer to causethe computer to perform: capturing notes of a user including a timestamp from the user associated with a meeting; synchronizing an agendaof the meeting and the notes of the user based on a correlation betweena time stamp of a topic on the agenda and a tune stamp of the notes ofthe user; and analyzing the synchronized topic and the notes todetermine highlights of the meeting based on a co-occurrence of the timestamp of the notes of the user and the time stamp of the topic on theagenda.
 9. The computer program product of claim 9, wherein theanalyzing further analyzes a concept of the co-occurring notes todetermine the highlights only if the concept of each of theco-occurrence of the time stamp of the agenda and the time stamp of thenotes have an overlap with each other greater than a predeterminedthreshold value.
 10. The computer program product of claim 9, furthercomprising creating a summary of the highlights of the meeting byaggregating the co-occurring notes for each of the plurality of users.11. The computer program product of claim 9, wherein the highlights aredetermined according only to a match of the time stamp of theco-occurrence of notes.
 12. The computer program product of claim 9,further comprising identifying a concept associated with each of thehighlights from processing a recorded audio of the meeting to create asummary of the meeting based on concepts.
 13. The computer programproduct of claim 9, wherein the co-occurrence is determined based on atleast two of the plurality of users taking notes at a same time stamp.14. A meeting summarization system, comprising; a processor; and amemory, the memory storing instructions to cause the processor toperform: capturing notes of a user including a time stamp from the userassociated with a meeting; synchronizing an agenda of the meeting andthe notes of the user based on a correlation between a time stamp of atopic on the agenda and a time stamp of the notes of the user; andanalyzing the synchronized topic and the notes to determine highlightsof the meeting based on a co-occurrence of the time stamp of the notesof the user and the time stamp of the topic on the agenda.
 15. Thesystem of claim 14, wherein the analyzing further analyzes a concept ofthe co-occurring notes to determine the highlights only if the conceptof each of the co-occurrence of the time stamp of the agenda and thetime stamp of the notes have an overlap with each other greater than apredetermined threshold value.
 16. The system of claim 14, furthercomprising creating a summary of the highlights of the meeting byaggregating the co-occurring notes for each of the plurality of users.17. The system of claim 14, wherein the highlights are determinedaccording only to a match of the time stamp of the co-occurrence ofnotes.
 18. The system of claim 14, further comprising identifying aconcept associated with each of the highlights from processing arecorded audio of the meeting to create a summary of the meeting basedon concepts.
 19. The system of claim 14, wherein the co-occurrence isdetermined based on at least two of the plurality of users taking notesat a same time stamp.
 20. The system of claim 14, embodied in acloud-computing environment.